DOCUMENT RESUME 



ED 465 758 


TM 033 897 


AUTHOR 


Jones, Timothy B.; Sherman, Ross B.; Ninness, H. A.; 
Hallman, Patsy J. 


TITLE 


The Relationship between GRE Scores and Performance on the 
Standard Principal Examination for Certification of 
Educators in Texas (ExCET) . 


PUB DATE 


2002-02-16 


NOTE 


2 Op . ; Paper presented at the Annual Meeting of the Southwest 
Educational Research Association (Austin, TX, February 
14-16, 2002) . 


PUB TYPE 


Reports - Research (143) -- Speeches/Meeting Papers (150) 


EDRS PRICE 


MF01/PC01 Plus Postage. 


DESCRIPTORS 


Certification; Correlation; educational Administration; 
*Graduate Students; Graduate Study; Higher Education; 
Prediction; Principals; Regression (Statistics) ; *Scores; 
Screening Tests; Test Results 


IDENTIFIERS 

ABSTRACT 


*Graduate Record Examinations; Scattergrams ; *Texas 
Graduate programs in Educational Administration use a 



variety of screening strategies, including the Graduate Records Examination 
(GRE) when selecting and admitting students. In order to earn Standard 
Principal Certification in Texas, an Educational Administration candidate, in 
addition to completing a graduate program, must also pass the Standard 
Principal Examination for the Certification of Teachers in Texas (ExCET) . 

This study examined the correlation between a students GRE score and the 
students ExCET score. Scores of 30 students from 1 university and 23 from 
another were used. The data clearly demonstrate a correlation between the GRE 
and ExCET scores. A scatterplot was derived to show this correlation, and a 
least square regression line was used to derive a linear equation for 
predicting Standard Principal ExCET score using the GRE score. (SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



TM033897 ed 465 758 



GRE Scores and ExCET Performance 1 



Running head: THE RELATIONSHIP BETWEEN GRE SCORES AND PERFORM 



The Relationship Between GRE Scores and Performance 

on the 

Standard Principal Examination for Certification of Educators in Texas (ExCET) 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 






Timothy B. Jones, Ed.D. 

Assistant Professor of Educational Administration 
Director, Southwest Educational Leadership Academy 
The University of Texas at Tyler 
3900 University Blvd. 

Tyler, Texas 75799 
903-566-7285 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
/ CENTER (ERIC) 

m This document has been reproduced as 
. received from the person or organization 
originating it. 



tiones@mail.uttyl.edu 



□ Minor changes have been made to 
improve reproduction quality. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



Ross B. Sherman, Ed.D. 
Professor of Educational Administration 
Chair, Department of Special Services 
The University of Texas at Tyler 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



H.A. (Chris) Ninness, Ph.D. 
Associate Professor of Human Services 
Stephen F. Austin State University 



Patsy J. Hallman, Ph.D. 
Interim Dean, College of Education 
Stephen F. Austin State University 



Paper presented at the Annual Conference of the Southwest Educational Research 
Association February 16, 2002 in Austin, Texas 




2 



GRE Scores and ExCET Performance 2 



w 



ABSTRACT 

Graduate programs in Educational Administration use a variety of screening 
strategies, including the Graduate Records Examination (GRE), when selecting and 
admitting students. In order to earn Standard Principal Certification in Texas, a 
Educational Administration candidate, in addition to completing a graduate program, 
must also pass the Standard Principal Examination for the Certification of Teachers in 
Texas (ExCET). This study examined the correlation between an Educational 
Administration student’s GRE score as a predictor of that students ExCET score. 

The data clearly demonstrates a correlation between the Graduate Records 
Examination (GRE) and the Standard Principal Examination for the Certification of 
Teachers in Texas (ExCET). Having identified this correlation, using a scatterplot, the 
study also utilized a least squares regression line in order to derive a linear equation 
for predicting Standard Principal ExCET score using GRE scores. 
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The Relationship Between GRE Scores and Performance 

on the 

Standard Principal Examination for Certification of Educators in Texas (ExCET) 

Educational Research Service (1998) has estimated that within the next ten 
years 50 percent of all public school administrators will retire. According to their 
report, Is There a Shortage of Qualified Candidates for Openings in the Principalship, 

47 percent of urban districts, 45 percent of suburban districts, and 52 percent of rural 
districts reported shortages of qualified candidates for principal vacancies. Who will 
fill this void? Will there be a sufficient quantity of qualified individuals to meet the 
demand? 

The certifying of professional administrators is a state decision. In the past the 
completion of a sequence of courses and submitting the appropriate application to a 
state agency charged with certifying administrators has been the requirement for 
certification as an administrator. However, as States have moved toward a 
performance based accountability model more and more states have instituted a 
examination as a part of the initial certification process. 

The Problem: GRE as a Predictor of Success on Standard Principal ExCET 

Historically one of the factors used to determine admittance to graduate schools 
is the Graduate Record Examination (GRE). However, is GRE performance a good 
predictor of success on the Standard Principal ExCET? The State of Texas uses a 
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performance based accountability model at all levels of its educational system. Since 
1981, one of the cornerstones of the system is the Examination for Certification of 
Educators in Texas (ExCET). Prior to receiving a certification in Texas, a candidate 
must secure a satisfactory score on the ExCET as established by the State Board for 
Educator Certification (SBEC). SBEC is the entity legislatively charged with 
overseeing all certification of educators in the State of Texas. The philosophy implied 
in the system is that certification assessments help to ensure that successful candidates 
possess the proficiencies necessary for success in the particular position. 

Certification - Standard Principal 

Chapter 241 of the Texas Administrative Code (2001) establishes the 
requirements for a Standard Principal certification, which is required for an 
administrative position as an assistant principal or principal in a Texas public school. 
Effective September 1 , 2000, prior to admission to an administrator preparation 
program, an individual must: 1) hold a baccalaureate degree from an accredited 
institution of higher education, 2) demonstrate an acceptable combination of a score on 
a national ly-normed assessment and grade point average determined by the 
preparation program. 

Standard Principal - ExCET 

Principal certification in Texas requires the certification candidate to earn a 
passing score on the Standard Principal ExCET (Number 68). The Standard Principal 
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ExCET content is derived from and tests knowledge, skills and abilities in nine 
competencies clustered around three domains. They are as follows: 

Domain I - School Community Leadership 

Competency 001 

The principal knows how to shape campus culture by facilitating the 
development, articulation, implementation, and stewardship of a vision of that 
is shared and supported by the school community (State Board for Educator 
Certification, 2000, p. 4). 

Competency 002 

The principal knows how to communicate and collaborate with all 
members of the community, respond to diverse interests and needs, and 
mobilize resources to promote student success (p. 7). 

Competency 003 

The principal knows how to act with integrity, fairness, and in an ethical 
and legal manner (p. 7). 

Domain II - Instructional Leadership 

Competency 004 

The principal knows how to facilitate the design and implementation of 
curricula and strategic plans that enhance teaching learning; ensure alignment 
of the curriculum, instruction, resources, and assessment; and promote the use 
of varied assessments to measure student performance (p.8). 
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Competency 005 

The principal knows how to advocate, nurture, and sustain an 
instructional program and a campus culture conducive to student learning and 
staff professional growth (p. 9). 

Competency 006 

The principal knows how to implement a staff evaluation and 
development system to improve the performance of all staff members, select 
and implement appropriate models of supervision and staff development, and 
apply the legal requirements for personnel management (p. 10). 

Competency 007 

The principal knows how to apply organizational, decision making, and 
problem solving skills to ensure effective learning environment (p. 10). 

Domain III - Administrative Leadership 

Competency 008 

The principal knows how to apply principles of effective leadership and 
management in relation to campus budgeting, personnel, resource utilization, 
financial management, and technology (p. 11). 

Competency 009 

The principal knows how to apply principles of leadership and 
management to campus physical plant and support systems to ensure a safe and 
effective learning environment (p. 11). 
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Standard Principal ExCET Development 

The Standard Principal ExCET was developed by soliciting input from a cross 
section of K-12 educators, faculty from educator preparation programs, education 
service center personnel, professional organizations, content experts, business 
community and parents. The ExCET development process included the following 
phases: 

1 . Developing standards for what beginning principals should know; 

2. Reviewing standards through public comment period; 

3. Developing the framework, which included the specific competencies; 

4. Conducting a content validation survey to determine the relative job 
importance of each competency; 

5. Developing and reviewing test item by the test developer; 

6. Conducting a pilot test of the new items; 

7. Reviewing the pilot test data and conducting preliminary standards setting; 

8. Administering the new ExCET Principal test; 

9. Setting the passing standards (State Board for Educator Certification, 

2000 ). 

Graduate Record Examination (GRE) 

Historically, one of the criteria used for screening and admitting a student to a 
graduate school is the use of the Graduate Record Examination (GRE) developed by 
the Educational Testing Service (2001a). This premise suggests a relationship between 



GRE Scores and ExCET Performance 8 



a particular score attained on the GRE and the likelihood of future success in graduate 
level courses. 

The Graduate Record Examination’s General Test measures an individual’s 
verbal, quantitative, and analytical skills that have been acquired over time and are not 
related to a specific field of study. The verbal component measures ability to analyze 
and evaluate material and synthesize information obtained from it. The quantitative ■ 
measures basic mathematical skills and the understanding of elementary mathematical 
concepts. The analytical test measures an ability to understand, deduce, analyze and 
evaluate information by identifying hypotheses, drawing conclusions, and developing 
plausible causal explanations (Educational Testing Service, 2001a). 

Each subtest yields a score between the range of 200 - 800. Scores from the 
different subtests should be considered independent of one another because each 
measure is scaled separately. For instance, based on all examinees who took the test 
between October 1, 1996 and September 30, 1999 a score of 600 on each of the 
subtests would placed the individual at the 85 percentile in verbal, 55 percentile in 
quantitative and 61 percentile in analytical. During the same period, the mean scores 
of all test takers were 471 verbal, 569 quantitative and 547 analytical. Whereas, the 
mean scores for examinees who reported education as there major were 447 verbal, 

5 1 1 quantitative and 534 analytical (Educational Testing Service, 2001b). 
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Considerations in Score Interpretation 

According to the Educational Testing Service (2001a), the Graduate Records 
Examination scores provide a common measure for comparing the abilities of students 
who come from a variety of colleges and universities with different standards and 
methods for measuring success in their program. However, it should be noted that the 
Educational Testing Service does not advocate using the GRE in isolation but as a 
supplement to other information such as undergraduate record, letters of 
recommendation, interviews, or other information used in the admissions process. 

Educational Testing Service (2001b) conducted a study during the 1993-1994 
testing year (N = 417K) to determine the effect on different subgroups when a 
combined verbal and quantitative score of 1000 was used for admission to graduate 
school. For instance 86 percent of African American, 69 percent Mexican American, 

61 percent of Hispanics, whereas, only 45 percent of whites and 38 percent of Asian 
Americans fall below the combined verbal and quantitative score of 1000. With regard 
to gender, 55 percent of females scored below the cut off as opposed to 35 percent of 
males. The conclusion of the study was that the use of adhering to a particular cut off 
score can systematically discount large numbers of students from various subgroups. 
Interestingly, Davidson and Lewis (2000) studied the predictive validity of GRE, IQ 
and general academic aptitude but failed to establish any correlation between GRE and 
criterion-referenced examinations . 
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Design 



The design for this study was correlational research. Students enrolled in a 
Standard Principal certification program at two regional universities served as the 
source of data. The study involved collection of two test scores from each participant 
in an effort to analyze any potential correlation and/or predictive value of one to the 
other. 

Research Questions 

Most accredited universities and graduate programs require some form of 
standardized test, such as the Graduate Records Examination (GRE), as a criteria for 
program admission. The two universities involved in this study use the Verbal and 
Quantitative scores on the GRE as part of their respective admission criteria. Given 
this fact, the research in this study was guided by the following questions: 

1 . Is there a correlation between the Graduate Records Examination score 
and the Standard Principal Examination for Certification of Educators in 
Texas score of students enrolled in a university administrator preparation 
program? 

2. Does the Combined Verbal/Quantitative Graduate Records Examination 
scores of students enrolled in a university administrator preparation 



program provide any predictive value for performance on the Standard 



Principal Examination for Certification of Educators in Texas? 
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A correlation, as reflected in the answer to the first research question, 
determined the applicability of the second research question. Absent an identified 
correlation between the two sets of scores, the predictability of such scores would be 
obviously negative. 

Instrumentation 

The scores from two instruments were used in this study. First, scores from the 
verbal and quantitative portions of the Graduate Records Examination were combined 
(referred to as Combined Verbal/Quantitative GRE Score) to make one score for each 
student in the study. Possible scores on each portion of the Graduate Records 
Examination range from 200 to 800 (Educational Testing Service, 2001a). The 
maximum score that could be earned on the combination of verbal and quantitative 
was 1600. 

Additionally, the total score earned by each student on the Examination for the 
Certification of Teachers (ExCET) Number 68, Standard Principal, was also used. 

The Standard Principal ExCET was revised by the State of Texas for use beginning 
with the Fall of 2000 test administration date( State Board for Educator Certification, 
2000). This revision included a change in content as well as a change in the 
methodology and scale used to score the instrument. While domain scores are 
calculated on ExCET Number 68, only the total scale score was used in this study. 

The possible scores on this instrument range from 100 to 300. 
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Sample 

The sample for this study consisted of students enrolled in one of two regional 
university administrator preparation program. The group from one university 
(University A) was composed of all students taking the Standard Principal ExCET 
during the first Spring 2001 test administration date. The group from the other 
university (University B) in the study was composed of all students taking the 
Standard Principal ExCET during the second Spring 2001 test administration date. 

The 30 students from the University A were added to the 23 students from 
University B which collectively created the sample for this study with 53 students. Of 
the 53 students making up the sample, 54.7 percent (n — 29) were female and 45.3 
percent (n = 24) were male. 

Ethnically, the study sample divided in to three groups. White students, with 
73.6 percent (n = 39) of the sample, made of the largest ethnic group. African- 
American students accounted for 17.0 percent (n = 9) for the study sample followed by 
Hispanic students at 9.4 percent (n = 5). 

Results 

The first research question was answered using the Pearson Product-Moment 
Coefficient (r) statistical treatment with a minimum r score of 0.65 required to indicate 
a correlation (Fraenkel & Wallen, 2000) between scores. Statistical significance for the 
correlation was established by the researchers at the 0.01 level. 
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The second research question was answered using a scatterplot and simple 
prediction equation. The prediction equation was calculated after applying a 
regression line to the scatterplot and expressing the line as an equation. 

Combined Verbal/Quantitative GRE Scores for the students enrolled at 
University A showed a low score of 5 10 and a high score of 1080. University A score 
analysis documents a mean of 796.33 for the Combined Verbal/Quantitative GRE 
Scores with a standard deviation of 161.79. University B Combined 
Verbal/Quantitative GRE scores showed a range with a minimum score of 580 and a 
maximum score of 1 170. The mean for University B scores was 854.78 with a 
standard deviation of 143.68. 

Standard Principal ExCET scores between the two University groups was somewhat 
similar. University A produced an ExCET score range of a minimum of 195 and a maximum 
of 284. The mean for the University A ExCET scores was 249.70 with a standard deviation of 
20.80. University B ExCET scores produced a minimum score of 217 and a maximum score 
of 279. The mean of the University B scores was 253.22 with a standard deviation of 14.43. 
The remaining data and statistical findings for this study are presented as they apply to each 
of the two research questions. 

Research Question One . Statistical analysis of the sample data of Combined 
Verbal/Quantitative GRE Scores and Standard Principal ExCET scores provide 
Pearson Product-Moment Coefficient (r) scores of 0.686, thus indicating the existence 
of a correlation between the GRE and ExCET scores. Statistical significance of this 
correlation was determined to be at the 0.01 level. 
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Research Question Two . With a confirmed correlation established between both 
sets of scores, further treatment was applied in order to assess the predictive value of 
the GRE score for performance on the ExCET. Upon plotting the scores on a 
scatterplot (see Figure 1), a line has been statistically charted to indicate the 
regression. The least squares regression line passes through the data according to the 
derived linear equation [y = 0.0768x + 187.92] whereas y is the predicted ExCET 
score and x represents the actual Combined Verbal/Quantitative GRE score. The 
standard error of slope was found to be 0.01 1289. 

In order to test the null hypothesis that the true slope is zero, the ratio of the 
obtained slope to its standard error was calculated. This calculation yielded a t-test 
statistic of [0.0768 / 0.01 1289 = 6.803] showing statistical significance (p < 0.00001). 
Thus, the coefficient for the GRE is significantly different from zero, and this provides 
substantial evidence that scores on ExCET Number 68, Standard Principal, are 
significantly influenced by GRE skills. Additionally, the coefficient of determination 
(r ) was calculated to be 0.471 1. The coefficient of determination indicates that about 
47percent of the variation in ExCET score performance is explained by performance in 
the Graduate Records Examination. 

Finally, the standard error of estimate, to indicate an index of prediction error, 
was determined to be 13.294. Given this value, an individual who obtains a mean 
combined GRE score of 1000 is predicted to score 265 on the Standard Principal 
ExCET. Moreover, 95 percent of the time, such a student will score between 238 and 
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291 on a similar test. Interestingly, this student has a 96.57 percent probability of 
obtaining a passing score (240) on the Standard Principal ExCET. 

A student who scores only 800 on the Combined Verbal/Quantitative GRE is 
predicted to score 249 on this ExCET. In this scenario, the 95 percent confidence 
interval concludes a Standard Principal ExCET score between 223 and 276, and the 
student has a 75.82 percent probability of passing the examination. The aggregated 
data displayed on Figure 1 seem to suggest that the probabilities for passing the 
Standard Principal ExCET decrease substantially when a student scores below 800 on 
the Combined Verbal/Quantitative Graduate Records Examination. 

Conclusion 

The data clearly documents the significant relationship and correlation of the 
Graduate Records Examination (GRE) and the Standard Principal Examination for the 
Certification of Teachers in Texas (ExCET). With such a relationship and correlation 
established, regression line derived from the data offers an important and helpful 
equation in predicting a Standard Principal ExCET score from a GRE score. This 
validates the use of the GRE to meet the state requirement that a standardized test be 
used in admission and screening standards for certification candidates. This also may 
assist university faculty, involved in Standard Principalship graduate certification 
programs, in the screening and admission process. 

While the sample did include students from two different Texas public 
institutions of higher education, the total number of students in the study (n=53) could 
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be expanded. Replication of this study incorporating data from additional Texas 
Universities would be useful and thus an implication for further study. Also, as the 
new Standard Principalship ExCET continues to be administered, additional data at the 
two Universities represented in the study and others will become more abundant thus 
offering additional opportunity for study. 
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